FlinkSQL Upsert/Retraction 写入 MySQL,Upsert 是什么意思?

您所在的位置:网站首页 flinkSQl insert FlinkSQL Upsert/Retraction 写入 MySQL,Upsert 是什么意思?

FlinkSQL Upsert/Retraction 写入 MySQL,Upsert 是什么意思?

2024-01-26 23:42| 来源: 网络整理| 查看: 265

如果我的sink是mysql 支持主键索引,我可不可以理解处理逻辑是retract 和upsert是一样的;上游数据false标记的是失效的记录,我删除失效的或者更新失效的数据是无区别的;

其实我还是对retract流和upsert流有点疑问 https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/dev/table/streaming/dynamic_tables.html

Append-only stream: A dynamic table that is only modified by INSERT changes can be converted into a stream by emitting the inserted rows.

Retract stream: A retract stream is a stream with two types of messages, add messages and retract messages. A dynamic table is converted into an retract stream by encoding an INSERT change as add message, a DELETE change as retract message, and an UPDATE change as a retract message for the updated (previous) row and an add message for the updating (new) row. The following figure visualizes the conversion of a dynamic table into a retract stream.

Upsert stream: An upsert stream is a stream with two types of messages, upsert messages and delete messages. A dynamic table that is converted into an upsert stream requires a (possibly composite) unique key. A dynamic table with unique key is converted into a stream by encoding INSERT and UPDATE changes as upsert messages and DELETE changes as delete messages. The stream consuming operator needs to be aware of the unique key attribute in order to apply messages correctly. The main difference to a retract stream is that UPDATE changes are encoded with a single message and hence more efficient. The following figure visualizes the conversion of a dynamic table into an upsert stream.

The API to convert a dynamic table into a DataStream is discussed on the Common Concepts page. Please note that only append and retract streams are supported when converting a dynamic table into a DataStream. The TableSink interface to emit a dynamic table to an external system are discussed on the TableSources and TableSinks page.

retract流做分组聚合,满足条件的group 字段数据来了,需要更新,发出一条false 标识的旧数据告诉下游这条数据失效了;

但是Upsert 是什么意思;An upsert stream is a stream with two types of messages, upsert messages and delete messages. A dynamic table that is converted into an upsert stream requires a (possibly composite) unique key. 有点不明白*来自志愿者整理的FLINK邮件归档



【本文地址】


今日新闻


推荐新闻


CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3